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ABSTRACT 


This thesis addresses the problem of estimating a random process from two 
observed signals sampled at different rates. The case where the low-rate observation has 
a higher signal-to-noise ratio than the high-rate observation is addressed. Both adaptive 
and non—adaptive filtering techniques are explored. For the non—adaptive case, a 
multirate version of the Wiener—Hopf optimal filter is used for estimation. Three forms 
of the filter are described. It is shown that using both observations with this filter 
achieves a lower mean—squared error than using either sequence alone. Furthermore, the 
amount of training data to solve for the filter weights is comparable to that needed when 
using either sequence alone. For the adaptive case, a multirate version of the LMS 
adaptive algorithm is developed. Both narrowband and broadband interference are 
removed using the algorithm in an adaptive noise cancellation scheme. The ability to 
remove interference at the high rate using observations taken at the low rate without the 


high-rate observations is demonstrated. 
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EXECUTIVE SUMMARY 


This thesis addresses the problem of estimating a random process using two 
observation sequences; one sequence is sampled at a lower rate than that of the estimated 
process. This has potential military applications in the areas of satellite-based remote 
sensing, network-based sensor suites, and various other multirate signal and image 
processing applications. 

Two new multirate filtering algorithms are described. The first multirate filtering 
algorithm is based on the Wiener—Hopf least squares optimal filtering equations. Results 
show that using both observation sequences with this filter provides a lower mean— 
squared error than when using a classic Wiener—Hopf filter with either the high-rate or 
low-rate observations alone. Additionally, the amount of training data needed to 
estimate the filter weights proved to be comparable to that needed when using either data 
sequence alone. The second multirate algorithm is adaptive and based on the least mean 
square (LMS) algorithm of Widrow and Hoff. Results are presented in terms of an 
adaptive noise cancellation scenario. The ability to remove both narrowband and 
broadband interference from a signal at the high rate using low-rate observations alone is 


demonstrated. 
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I. INTRODUCTION 


Multirate signal processing has become an important area of digital signal 
processing since there are few standards that govern the rate at which data is collected 
and sampled. Multirate signal processing is a rich field for research, encompassing 
everything from deterministic operations like sampling rate conversion to statistical 
treatments using multiple observations. Some applications include sampling rate 
conversion for oversampling subsystems for CD or DAT players [1], and subband coding 
of speech in digital communications systems [2]. Some statistical research involves 
wavelets for modulating signals to be transmitted over channels with unknown 


characteristics [3]. 


This thesis describes research which is part of an overall project to investigate 
methods of combining information taken from sensors with different sampling rates. 
Specifically, this thesis addresses the situation where an underlying continuous—time 
signal is measured by multiple sensors, each with a different sampling rate, and a 


different signal-to-noise ratio. 


The basis for this research was established previously [4]. The purpose of this 
work was to 1) investigate and validate the previous theoretical work with simulation 
results; 2) extend the methods using a least squares approach, and 3) extend the methods 


to adaptive filtering. 


The organization of this thesis is depicted in Figure 1. The thesis consists of four 
chapters and an appendix. Chapter I introduces the topic and provides a description of 
the filtering problem. Chapters II and III address related but separate problems. Chapter 
II develops a multirate form of the Wiener—Hopf equation for optimal filtering using least 
squares methods and presents results using this filter. Chapter III describes an adaptive 
multirate filter and shows results in the context of an adaptive noise cancellation 
algorithm. Chapter IV presents conclusions and recommendations for future studies. 


The Appendix provides details about the various test data used in this work. 


CH I: Introduction 


CH Il: The Multirate Wiener Filter 


| A: Problem Description 


CH Ill: The Multirate LMS Filter 


| B: Forms of the Filter A: Background 
1: Direct Form 2: Innovations Form 3: Interpolation Form 


C: Computation and Training B: Filter Description 


! 


C: Simulation Preliminaries 


D: Simulation Results | 
D: Simulation Results 


1: Single vs. Multirate | 


1: Broadband Interference 2: Narrowband Interference 


2: Required Training Data 


| 


3: Weight Optimization 


CH IV: Conclusions and Recommendations 


Figure 1. Thesis Outline Flow Diagram. 


I. THE MULTIRATE WIENER FILTER 


This chapter discusses three (non—adaptive) forms of the optimal multirate filter. 
Quantitative results comparing the single—rate and multirate Wiener filters are given. 
The amount of training data required to estimate the filter coefficients is addressed. In 


particular, the results of experiments using various lengths of training data are presented. 


A. PROBLEM DESCRIPTION 


The problem to be considered here is as follows. Given sensor data sampled at 
different rates and with different signal—to—noise ratios (SNR), optimally combine the 
data to form an estimate of the original signal. The system—level diagram for the case of 


two sensors is depicted in Figure 2. 


7,(7] 
,| Sensor 1 ela 
(R Khz) , 
Optimal d[n] 
Multirate ;———> 
Filter 
,| Sensor 2 yl] 
(R/K Khz) 
7,[m] 


Figure 2. | Overview of Estimation Process. 


An underlying continuous-time signal d(t) is monitored by two sensors which 
produce noisy observation sequences. The noise sequences are represented by 77,[n] and 
7,[m] and are assumed to be additive. Different discrete time indices ‘n’ and ‘m’ are 


3 


used to indicate the different sampling rates. One sensor operates at a sampling rate of R 
kHz while the other samples at R/K kHz; the faster sampling rate is thus an integer 
multiple of the slower one. The factor K is the ratio between the two sensor data rates. 
Both data sequences are fed into a multirate filter where they are used to jointly produce 
an optimal estimate d [n] of the original signal d(t). The estimate is needed or desired to 


be computed at the higher rate. Table 1 lists the notation used throughout this thesis. 


Symbol Meaning 

x[n] The data sequence which is sampled at the higher rate. 

y[m] The data sequence which is sampled at the lower rate. 

d[n]| The desired signal to be estimated. 

d[n] The estimate of the underlying signal d[n]. 

P The order of the filter that operates on the high-rate data x[n]. 
O The order of the filter that operates on the low-rate data y[m]. 
K The ratio between the high and low sampling rates. 

SNR ich The signal-to-noise ratio of the high-rate data sequence x[7]. 
SNR, The signal—to-noise ratio of the low-rate data sequence y[m]. 
N The length of the high-rate data sequence x[n]. 

M The length of the low-rate data sequence y[m]. 

= Notation denoting the least squares solution to the equation. 

n The discrete time index of the high-rate data sequence. 

m The discrete time index of the low-rate data sequence. 

E {e} The expected value of the expression inside the braces. 

MSE The mean—squared error between the desired signal and estimate. 


Table 1. List of Notation. 


B. THREE FORMS OF THE FILTER 


Three possible forms of the filter were considered in this thesis. These are 


discussed separately below. 


1. Direct Form 


The direct form is the simplest and the most basic form of the filter; hence the 


name. Figure 3 illustrates the structure of this form of the optimal multirate filter. 


y d[n| 
db 


Figure 3. Direct Form of the Multirate Wiener Filter. 


The two noisy sensor observation sequences x[n] and y[m] are simultaneously 
fed into separate time—varying linear filters. The filter h,[n] operates at the sampling 
rate of x[n] while g,[m] operates at the sampling rate of y[m]. The two outputs are 


summed to produce the estimate d[n]. The filters are to be chosen to minimize the 
A 2 
mean—squared error ( MSE) defined as E ‘(a - d{n]) } . 
The simultaneous processing of data by these two filters leads to a periodic time 


dependency in the filter parameters. This is illustrated in Figure 4 and can be explained 


as follows. 


Step ; Haier OS 0O2 Illustration 
High-rate | Low-rate 
No 
[@ e e i | e ° e e 
0 Ny My Le I ° 
My 
n+l 
® e e e j ° e e 
1 yl My e 1 e 
Mo 
ny+2 
e e e e e ! | e e 
p) Nye d M9 ° ! ° 
My 
ny+3 
e ® e { @ e e ! } e 
3 Nat 3 m,+1 ° [e t 
m,t+1 


Figure 4. _ Internal Operation of Multirate Wiener Filter. 


Recall that the time index of the high-rate filter is n while the time index of the low-rate 
filter is m. The estimate d is to be produced at the high (full) rate. For the case 
illustrated in Figure 4, the order of the high-rate filter P is 4, the order of the low-rate 
filter O is 2, and the ratio of sampling rates K is 3. The filters 4,[n] and g,[m] are 
positioned at the corresponding starting points n, and m, in their respective data 
sequences. The estimate d[n] is obtained with the filters in these positions. This is step 
zero in Figure 4. At step one, d[n, +1] (i.e., the estimate for the following point of the 
high-rate data sequence) is found by sliding the high-rate filter h,[n] forward by one 
point while the low-rate filter remains in place. This is repeated for the estimate 

dn, + 2], shown in step two in Figure 4. Finally at step three, the estimate dn, + 3] is 


found by sliding both filters forward by one point in their respective sequences. Note that 
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at this step the filters are now in the same relative positions as the initial step and the 
process can be repeated. This continues for the entire length of the available data 
sequences. This pattern for processing the data is what causes the optimal filters to be 
linear periodically time—varying (LPTV). In general, the filters have K unique positions 
relative to each other. Each unique position contains a different set of data points and 
requires unique sets of filter coefficients for the estimate; hence (P+ Q)K filter 


parameters are needed to specify this filter. 


a. Derivation of Estimate 


The estimate of the desired signal can be written as: 


é P-1 Q-1 
d{n]= >h,{ilx{n -i]+ Dg,Lily[m —/]. (1) 


i=0 j=0 


The time sample 1 can be written as 
n=Km+k; k=n(modk). (2) 


The subscript & on the filter weights is to indicate that the filters are periodically time— 
varying with period K. If the filter weights used at the K” step are written as 
h,=[A,(0] All] -- ALP-y, (3) 


and 
g.=[g[0] gf] -- gfO-} (4) 

while the observation vectors are defined as 
x[n]=[x[7] x[n-1] «+ xfn—-(P-D]] (5) 


and 


T 


y[m]=[ylm] ylm-1] + ylm-(Q-DI]. (6) 


Then (1) can be written as 


d{nj=h;x{n]+gry[m]; k=n(mod K). (7) 


b. Least Squares Formulation 


Least squares methods offer a convenient and data-dependent way to 
solve for the filter parameters. In order to pose the problem in terms of least squares 


methods, define the data vector 
d=[d[n,] [n+l] = d[n,+(N-DI] (8) 


where nz, is the initial point of the estimate and N is the number of samples. Define the 


data matrices as: 


x’ [1] y'[m)] 
x= x Pot Y- y eS (9) 
x'[n, +(N -)] y'[m +(M -1)] 


where n, and m, are the corresponding starting points in the observations sequences and 


N=KM. Following the notation in [6] we can write 


h, ls 
[X, v1 | |ta k =n(modK) (10) 


where the vector of values to be estimated d, is formed from the data vector d by taking 
every K” element beginning with element k+1. Likewise, X, is formed from X by 

Is 
taking every K" row of X beginning with row k +1. The notation = denotes a least 


squares solution which minimizes the squared error between the left and right sides of the 


equation. The solution is well known and is given by 


nl = [X, Y]'d, &=n(modXK) (11) 


where ‘+’ denotes the Moore—Penrose pseudoinverse of the matrix [X, Y]. The mean— 


squared error is given by: 


N-1 . 
MSE = 4S atn\(aln] - d[n}) (12) 
N n=0 
which after substituting Equation (2) and (6) can be written as 


MSE = >» d{Km+k](d[Km+k]—hjx[Km-+ k]- giylm]) (13) 


k=0 m=0 


This last equation can be written in vector and matrix notation as 
MSE = ii d,—d;"X,h, -d,"Yg, ) (14) 
n=0 


where d, and X, are formed as in the discussion following Equation (10). 


2. Innovations Form 


The innovations form of the filter, shown in Figure 5, explicitly shows how the 
low-rate observation sequence contributes to the estimate. The filter A°[n] in the top 
branch is time-invariant and is the optimal filter that would be used if the low-rate data 
were not present. The filter H,[”] is used to predict the low-rate data from the high-rate 
data. The prediction error sequence, which is the innovations sequence, is sent to the 
filter in the lower branch. Consequently, the output of the lower branch constitutes the 
additional information provided by the low-rate observations. The explicit handling of 
this new information is the reason the name ‘innovations’ is used for this form of the 


filter. 


by 
aaa | 
= 
= 
Vv 


h'[n] 


H,{n| Ow 


ylm] ms) > g{m] 


Figure 5. Innovations Form of Multirate Wiener Filter. 


This form of the filter can be derived as follows. Using some results from the 


theory of generalized inverses [8], Equation (10) can be written in partitioned form as 


ie =[X, Y]'d, 


gi 
xX -XvC" 
-| k a la (15) 
_ [| Xtd, -XtYC*d, 
C'd, 


where C= (i —X,X;, yy. From the above equation it can be seen that, if H, is defined as 
H, = X,Y, then the high-rate part of the filter can be written as 


h, =h;-H,g, (16) 


where h? = X;d, is the optimal filter when estimating the data using only the high-rate 

observations. In a stochastic process framework, this filter would not be a function of k 
due to the stationarity of the data [4]. However in this least squares framework, the filter 
does depend on k, with h? converging to a common value for long data sequences. The 


filter estimate can now be written as 
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d[n] = he’ x[n]-g, H,x[n]+ gl yf] 


(17) 
=hi"x[n]+g; (ylm]-H;x[7]) . 


This produces the form of the filter shown in Figure 5. 


J: Interpolation Form 


This form of the filter avoids the need for time—varying filters to process the data 
sequences. A diagram is shown in Figure 6. In this form, the low-rate data is inter— 


polated to the high-rate. Both data sequences are then processed by time-invariant 


filters. 


Figure 6. Interpolation Form of Multirate Wiener Filter. 


In this realization, all of the branch filters are time-invariant. This filter works as 
long as the input is stationary. This can be achieved if the low-rate data and correspond— 
ing interpolation filter are bandlimited to +7/K [5]. However the ideal interpolation 
filter is non—causal and, so, a causal approximation has to be used for this form. This 


may require the causal approximation filter to be of very high order. 


Cc. COMPUTATIONAL REQUIREMENTS 


In comparing the various realizations of the optimal multirate filter, a matter of 


concern is the number of filter parameters required as well as the number of operations 
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(multiplications and additions) at each time step. Table 2 lists these quantities for each 


form of the filter. 


Form of Optimal Filter | Filter Parameters | Operations / Unit Time 
Direct (P+ O)K P+O 
Innovations P+PQOK+QK P+PQ+@Q 
Interpolation P+QOK P+OK+I 


Table 2. | Computational Requirements for the Three Forms of the Filter. 


The direct form of the multirate filter, being the /east computationally expensive 
of the three forms, requires (P+ Q)K coefficients. A corresponding single—rate filter 
would require only P coefficients, which differs by approximately a factor of K. Notice 
however that the number of operations per unit time is P + Q which does not depend on 
K. This means that the computational requirement at each time step is the same as a 


filter with a fixed set of coefficients (i.e., a linear time—invariant filter). 


The innovations form is the most computationally expensive of the three forms of 
the filter due to the prediction filter H,. There are POK more filter parameters than for 
the interpolation form due to the prediction process. Additionally, the prediction filter 
results in PO more operations than needed using the direct form. This form would not 
likely be used in practice due to the extra computational cost over the direct form. This 
form is useful, however, in analyzing the reduction in MSE due to the presence of the 


low-rate observations. 
The interpolation form of the filter requires the fewest number of parameters 
P+QK (the filter g[n] is assumed to be of order OK since the low-rate data is 


interpolated by K up to the high rate). The number of operations is thus dependent on 
K and also depends on the order J of the interpolation filter, which can become the 


dominant part of the computation. 
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The length of data needed in solving for these parameters is a valid practical 


concern, and is addressed in the next section. 


D. SIMULATION RESULTS 


1. Preliminaries 


Since MSE was used as the criterion for the development of the filters, it was also 
used to measure filter performance in this study. The mean—squared error in decibels 
(MSE ,,), defined as MSE ,, =10log,, MSE, is used in the following discussion. The 
data used for these experiments is described in the Appendix. 


2s Multirate vs. Single—rate Filter 


In comparing the performance of the multirate and single—rate filters, a reasonable 
question to ask is “How much is performance improved (if at all) over using either of the 
data sequences alone?” This question can be answered by comparing the performance of 
the multirate filter to that of a single-rate Wiener filter on either of the data sequences 
separately. In the following experiment, the MSE was calculated while varying the order 


of the low-rate filter. The results are shown in Tables 3 and 4. 


For the simulations described here, the order of the high-rate filter P was 30 and 
the ratio of sampling rates K was 10. The SNR of the high-rate sequence was 0 dB. 
The SNR of the low-rate data sequence was 10 dB. The SNRs were obtained by adding 
channel noise to each of the observation sequences. The data were split into two sets. 
One set of data, called the ‘training set,’ was used to design the filter. The filter was then 
applied to the second set of data, which was called the ‘test set.” The length of both the 
training and test data sequences was 25,000 points. The use of both a training and test set 
helps judge performance of the filter with different, but statistically similar, data. The 
MSE on both the training and test sets was recorded; then the experiment was repeated 
using different realizations of sensor noise. The results were averaged over 100 trials of 


sensor noise. 


Case[1]: The order of the low-rate filter was taken to be Q=3. This means that 
the low-rate filter uses points covering the same time frame as the high-rate filter. Table 


3 shows that using the high-rate observations alone leads to an MSE of approximately 
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9.8 dB on both training and test data. The low-rate observations, when used alone, result 
in a MSE of approximately 13.5 dB which is worse by about 3.7 dB. When both data 
sets are used together however, the MSE is about 7.6 dB which is about 2.2 dB better 


than using the high-rate data alone. 


Mean-Squared Error 
Sensor 
Training Set (dB) | Test Set (dB) 
High-rate 9.84 9.86 
Low-rate 13.59 13253 
Both 7.65 7.64 


Table 3. | Mean—Squared Error for Q =3 (100 trial average). 


Case[2]: The order of the low-rate filter was taken to be Q=10. This means that 
the low-rate filter uses points covering a much larger time frame than the high-rate filter. 
Table 4 shows that the MSE for the high-rate data alone remains at approximately 9.8 dB 
as before. The MSE for the low-rate data alone improves (compared to the previous 


experiment) to about 10.5 dB. The use of both data sets together in this experiment 


Mean-Squared Error 
Sensor 
Training Set (dB) | Test Set (dB) 
High-rate 9.84 9.86 
Low-rate 10.67 10.53 
Both 5.96 a As 


Table 4. | Mean—Squared Error for Q = 10 (100 trial average). 
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results in a very significant improvement in performance. Specifically there is almost a 4 
dB reduction in MSE over using the high-rate data alone and about 3 dB over using the 


low-rate data alone. 


3. Required Training Data 


The previous section discussed the computational requirements of the filter. 
Another matter of practical concern is the amount of training data needed to design the 
multirate filter. The length of data needed to solve for these parameters is investigated 


here. The following results are based on the same data used in the previous subsection. 


For this experiment, the length of data used to design the filter was varied from 
500 to 25,000 points in steps of 500 points. The other factors are as stated above and are 
reproduced here for continuity: the order of the low-rate filter P was 30, the ratio of 


sampling rates K was 10, the SNR of the high-rate sequence SNK,,,,, was 0 dB, and the 


SNR of the low-rate sequence SNR,,, was 10 dB. The error MSE, on both the training 


low 
and test sets were computed and plotted. The length of data required for training was 
defined to be the length at which the relative error between the training and test sets came 


to within one percent. The results were averaged over 50 trials of sensor noise. 


a. Low-rate Filter Order QO =3 


The figure shows plots of MSE vs. length of data set. The top pair of 
curves result when using low-rate data alone; the middle pair is for the high—rate data 
alone, while the bottom pair results when using both data sets. For each case, the dotted 
lines correspond to the training sets, and the solid lines to the test sets. The point at 
which the MSE on the training and test sets come to within one percent of each other is 
circled. Using low-rate data alone, it took 10,000 points of training data to reach the 


criterion point. The case of a low-rate filter order QO = 3 is depicted in Figure 7. 
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Figure 7. | Mean—Squared Error versus Data Length for Q=3. TOP: Low-rate data alone. 
MIDDLE: High-rate data alone. BOTTOM: Both data sets. 


Using high-rate data alone, only 4,500 points were needed. Interestingly, using both data 
sets, only 7,000 points were needed. This is not an unreasonably high cost to design the 


multirate Wiener filter. 


b. Low-rate Filter Order 0 =10 


The above simulation was repeated using a low-rate filter order of O=10. 
Figure 8 shows the results graphically. Using low-rate data alone required over 25,000 
points of data. Designing the filter using both observation sequences, however, required 
only 4,500 points of data. This turned out to be exactly the same as the number required 


for using the high-rate data alone. 
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Figure 8. | Mean—Squared Error versus Data Length for Q=10. TOP: Low-tate data alone. 
MIDDLE: High-trate data alone. BOTTOM: Both data sets. 


The results of these experiments indicate that the length of training data needed to 
estimate the filter coefficients is comparable to that when using time-invariant filters on 


either sequence alone. 


4. Optimizing Filter Tap Weights 

As part of this research, a parametric representation of the filter impulse response 
was sought to help to reduce the number of weights to compute. This representation 
would exploit possible patterns in the filter coefficients. It is the time—variant filters that 
cause the computations to increase. Therefore, the time—variant behavior of the filters 


was examined for a pattern to exploit. 
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The FM, AR, and SINE signals (see Appendix) were used for this experiment. 
For each of the signals, the high-rate and low-rate filters were examined for patterns. 
The time-varying nature of the filter coefficients, however, proved to be signal— 


dependent. 


For example, a signal of length Z = 27,000 points was used to design a multirate 
Wiener filter with two observations sequences under the following conditions: P =100, 


Q=100, K =20, SNR,,., =0 dB, and SNR,,, =10 dB. These conditions yield 


low 


(100 +100)20 = 4000 filter weights (see Table 2). The filter coefficients h,[”] and 
g,[m] are plotted in Figure 9. Three—dimensional plots of the filter coefficients for the 


FM, AR, and SINE signals are shown. The left column of coefficients is for the high-rate 
filters, while the right column holds the low-rate coefficients. The time—varying axis is 
labeled ‘k’ denoting the cycle of the filter. The axis for the filter index is labeled ‘n’ or 
‘m’ depending on whether it shows the index for high-rate or low-rate filter, 


respectively. 


In analyzing Figure 9, we begin by considering the high-rate, time—variant filters 
(specifically, for the FM signal (Figure 9(a)), which resembles a two—dimensional wave). 
It seems that along the k axis, the filter values remain relatively constant with only minor 
fluctuations. This would also appear to be the case for the high-rate coefficients for the 
AR signal (Figure 9(c)) and SINE signal (Figure 9(e)). In addition, note that there is no 
general trend where the filter coefficients become small along the n or m direction, 
except for the case of the AR signal. A similar analysis applies to the low-rate filters. 
Begin with the low-rate filters for the FM signal (Figure 9(b)). There is no constant 
behavior along the time—varying axis. The low-rate coefficients of the AR signal (Figure 


9(d)) and SINE signal (Figure 9(f)) exhibit similar behavior. 


Two things are of note: 1) the constant behavior of the high-rate filters along the 
time-varying axis seems to validate the discussion of the innovations form of the filter. 
Specifically, the claim that the h; converge to a common value for long data sequences is 


supported; and 2) the filters appear to be unique for each type of signal estimated. 
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Figure 9. 
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(e) (f) 


Sample time—varying filters used in multirate Wiener filtering. (a) FM Signal: 


High-rate weights, (b) FM Signal: Low-rate weights. (c) AR Signal: High—rate weights, 


(d) AR Signal: Low-rate weights. (e) SINE signal: High-rate weights, 
(f) SINE signal: Low-trate weights. 
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The foregoing results covered the non—adaptive forms of the filter. The next 
chapter introduces an adaptive form of the multirate filter, and provides results in the 


context of adaptive interference cancellation. 
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Ul. THE MULTIRATE LMS FILTER 


This chapter presents an adaptive filtering method useful for applications where 
the statistics of the data are non-stationary. The algorithm is based on the least mean— 
squares (LMS) algorithm of Widrow and Hoff [9]. This application of the algorithm is in 
the context of interference removal. Results are given for broadband and narrowband 


interference in an adaptive noise cancellation (ANC) scenario. 


A. BACKGROUND 


Adaptive filtering is a wide discipline of which LMS and its variants are a 
significant part. In an adaptive filtering algorithm the data to be filtered is employed to 
find the optimum filter tap weights. 


The basic, single-rate LMS algorithm offers a low cost, elegant solution to 
adaptive filtering. An arbitrary initial set of filter coefficients is chosen and updated at 
each time step n and new observation. If the data is wide—sense stationary and the 
algorithm step size parameter (see below) is chosen appropriately, the filter coefficients 
will approach the optimal coefficients defined by the Wiener—Hopf equations. The 
nature of this convergence is discussed in many places [7, 12]. If the data is non— 
stationary and slowly time—varying, the filter coefficients tend to ‘track’ the optimal 
time—varying filter for the data. In the basic LMS method the filter coefficient vector w 


is updated at each time step n according to the equation [6] 
w[n + 1] = win] + we[n]x[n] (18) 


where 
e[n] = d[n]—w' [n]x[n] (19) 


is the error found at time step 1 and w is a parameter called the ‘step size.’ For suitably 
chosen values of the step size, the weights will converge to a solution close to that of the 


optimal (Wiener) filter. For practical purposes, the bounds on the step size are [7] 
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<< (20) 


PR, [0] 


where P is the order of the filter and R,[0] is the value of the autocorrelation sequence 


of the input signal at lag zero (1.e., the signal power). 


B. FILTER DESCRIPTION 


The multirate LMS estimation algorithm extends the LMS algorithm to the case 
where multiple inputs at different sampling rates are available. Accordingly for the 
multirate case, the equations are slightly more complicated. In this discussion it is 
assumed, as in the previous chapter, that two inputs are available (see Figure 10) and 


these are to be used jointly to estimate the ‘desired’ signal d[n]. 


Figure 10. Simplified Diagram of Multirate LMS Filter. 


The input x is sampled at the full rate, i.e., the rate of the output estimate. The input y is 
sampled at a rate of 1/K times the full rate. Again at any time 7, one can write 


n=Km+k for k=0,l,....K —1. Therefore, define a vector of high-rate data points as 
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x[n]=[x[n] x[n-1] + x[n-(P-D] (21) 
and a corresponding vector of low-rate data points as 
ylm]=[y[m] ylm-1] ++ ylm-(Q-DIJ’. (22) 
The estimate is then of the form 
d{n]=h'x[n]+g7y[m]; n=Km+k. (23) 


In the absence of adaptation the filter coefficients would be periodic, 1.e., at any 
times n, and n, such that n,(mod K)=n,(mod K) =k the coefficient vectors are h, and 
g,. For the multirate LMS algorithm these coefficient vectors are updated in time. The 
coefficient vectors at any time n will be denoted by h,[m] and g,[m] (where 


n=Km+k). The estimate is thus given by 
d{n]=hj[m]x(n]+ gi [mly[m] ; k=n(mod K). (24) 
The error is then given by 
e[n] = d[n]—d[r]. (25) 
The update steps in the multirate LMS algorithm are defined by 
h,[m+1]]=h,[m]+ 4,e[n]x[7] (26) 
and 
g,.[m+l]=¢,[m]+ “,eln]ylm]. (27) 


The complete algorithm is more easily specified with a double index. It is listed 


in this way in Table 5 and depicted graphically in Figure 11. 
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Step Equation 
(a) | x,liJ=[x[n,+iK+é] x[n)+iK+k-1] + x[n)t+iK+k-P+l]] 
yli]=[ylm, +i] yim +i-l] + ym, +i-O+I]] 
(b) d,{i] =, [i]x, [4] + gf yl] 
(c) d,{i] = d[n, +iK +k] 
(d) e,lil=d,[i]-d,[i] 
(e) h,[i+1]=h,[i]+ “,e,[i]x, [i] 
g [i+ =¢,li}+4,e[iyt 


Table 5. —_ Listing of the Multirate LMS algorithm. 
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Figure 11. Detailed Diagram of Multirate LMS Algorithm. 


The equations in Table 5 require some explanation. The time—varying data filters 


have period K, so there are PK weights needed for the high-rate filter and OK weights 
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needed for the low-rate filter. For the high—rate case, if we define a P by K matrix 


containing all high-rate coefficients 
Hf]=/bof] bl] bf] (28) 


whose columns represent the filter at each step k =0, 1, ..., K—1 in its period, then we 
see that only one column of H[Z] is updated for each point of the output sequence that is 
estimated (see Table 5). Beginning the estimation at corresponding points n, and m, of 
the observation sequences, the filter h,[0] is found. Then the input matrix x,[i] is 
updated and used to find h,[0]. The process continues until all K columns of H[i] have 
been updated, at which time the cycle repeats. The coefficients for the k” step in the 
cycle of the time-varying filter h, are therefore updated at every K” point of the original 


sequence. 


A similar discussion applies to the low-rate coefficients. The complete set of 


coefficients are contained ina QO by K matrix 
Gi=) el] gf) sgl]. (29) 
A particular column g, is then updated at every K“ point of the original data sequence. 


C. SIMULATION PRELIMINARIES 


The multirate LMS filter was tested in the context of adaptive interference 
cancellation, also known as adaptive noise cancellation (ANC). Results are given for 
both narrowband and broadband interference. For the basic, single-rate LMS ANC the 
primary input contains the signal plus interference. The so-called ‘reference’ input 
contains a correlated version of the interference that the adaptive filter uses to cancel the 


interference in the primary input. 


25 


Interf Cleaned 
nterference 
nn] Primary Input Signal 
= d 
Signal d[n] s[n]= d[n]+ nn] : [7] . 
(high rate) 
mn] 
Reference Inputs f 
Correlated _, See: 
Interference _lal= mila) a 
high rat 
one) Multirate LMS Filter 
Correlated —_y[m]=77;[m]| (estimate interference) 
Interference ———————» 
(low rate) 
Figure 12. Adaptive Interference Cancellation with Two Reference Inputs. 


The ANC extended to the case of multiple reference inputs is shown in Figure 12. 
The inputs to the filter now contain multiple versions of the interference, sampled at 
different rates. The goal of the LMS filter is to estimate the interference present in the 
reference input (at the full rate) from two noisy independent observations of the inter— 


ference. A case of some interest is the case where the only reference signal is y[m]. In 


other words, the reference signal is at a lower rate compared to the primary input s[n] 


and the cleaned signal d [n]. In this case it may be able to perform the ANC with fewer 


samples (in time) of the correlated interference. 
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D. SIMULATION RESULTS 


Results for both narrowband and broadband interference are presented in this 


section. 


1. Narrowband Interference 


In this experiment the interference consisted of tonal ‘noise’ added to recorded 
speech. Two sinusoidal tones of frequency 4410 and 8820 Hz were added to a segment 
of speech. The first tone was added at the start of the speech signal, and the second tone 
added in the middle of the sequence. The original speech signal and corrupted version 


are shown in Figure 13 below. 
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Figure 13. Plot of voice signal of the spoken word ‘hello’ used in narrowband interference 
simulation. (a) Original speech signal. (b) Speech signal including tones. 
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The test signal for this experiment was a recording of the spoken word ‘hello’ 
with a prolonged ‘o.’ The sample of speech was two seconds long and sampled at 22.05 
kHz. The waveform is shown in Figure 13(a). Two sinusoidal tones were added to the 
signal. A tone of 4.41 kHz was added at the beginning of the speech segment. At the 
midpoint of the speech segment, this first tone was ended and a second tone of 2.205 kHz 
was added for the remainder of the speech signal. The reference input was generated by 
sending the interference signal through a third-order FIR filter and downsampling the 
result (see Appendix). The corrupted signal is show in Figure 13(b) and illustrates the 


adverse effect of the interference on signal quality. 


The parameters chosen were as follows. The high-rate filter order P was 2, the 


low-rate filter order Q was 2 , the SNR of the high-rate sequence SNR,,,,, was 0 dB, and 


the SNR of the low-rate sequence SNR, 


»» was 10 dB. The number of samples required 
for the filter to mute the initial sinusoid is used as a measure of speed of convergence 
below. The original speech signal is reproduced in Figure 14(a) for comparison with the 


following results. 


a. Using Both Observation Signals 


The corrupted signal was filtered using the parameters described above; a 
sampling rate ratio of K = 4 was used. The resulting MSE was —14.15 dB. It took 
approximately 4,575 samples to mute the initial sinusoid in this case. For the given 
sampling rate of 22.05 kHz, this corresponds to only 0.21 seconds. This is slow for 
channel equalization in a communications system, but may suffice for certain audio 
applications. The filtered signal is shown in Figure 14(b). The spikes in the sequence at 


samples zero and 25,000 are the points at which the sinusoids were added. 


b. Using Low-Rate Observations Alone 


In this case, the corrupted signal is filtered using the low-rate observation 
sequence as the only reference input. The low-rate observation sequence has a sampling 


rate of 5.51 kHz. The multirate LMS filter estimates the interference at the full rate of 
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22.05 kHz, using these samples. All other simulation parameters are the same as the 
previous case. The filtered signal is shown in Figure 14(c). For this case, the filter took 


approximately 6550 samples (0.3 sec) to mute the initial sinusoid. The resulting MSE 
was —12.91 dB. 


This experiment was also performed using a longer segment of speech 


taken from the cockpit of a military jet; similar results were obtained. 
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Signals after filtering. (a) Original speech signal. (b) Filtered signal using both 
data sequences with K =4. (c) Filtered signal using low-rate data alone with K = 4. 


Figure 14. 


2s Broadband Interference 


An adaptive interference scenario was also used for this scenario. The desired 
signal for this experiment was a segment of radio traffic from an air traffic control center. 


The interference was a spoken phrase of speech, which is a broadband signal. The 
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reference input was generated by sending the interference signal through an FIR filter 
(see Appendix) and then downsampling the resulting sequence by a factor of K. The 
interference was removed using the low-rate observations alone. The parameters for this 
simulation were: the order of the low-rate filter Q = 40, the low-rate SNR was 


SNR,,,, =—0.26 dB, the sampling rate ratio was K = 4. The SNR of the filtered sequence 


was 9.65 dB, which is an improvement of 9.91 dB. The supporting plots are shown in 
Figure 15. The original speech signal and is shown in Figure 15(a) and exhibits some 
clipping. The speech corrupted with interference is shown in Figure 15(b). The filtered 
signal is shown in Figure 15(c). Figures 15(d) and (e) show the actual and estimated 


interference signal, respectively. 


The foregoing discussion constitutes the penultimate results of this thesis. The 
next chapter summarizes the conclusions of this research and offers suggestions for future 


work. 
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Figure 15. Results for multirate LMS filtering using low-rate observations alone. (a) 
Original speech signal. (b) Signal plus interference. (c) Filtered signal. (d) Original 
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IV. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 


This work developed 1) performance results for the multirate optimal filter and 
multirate LMS adaptive filters, 2) a least squares approach to finding the filter 
coefficients, and 3) an adaptive form of the filter based on the LMS algorithm. Both of 
these filters were used to solve an estimation problem in which multiple independent 
observations of the data were available. This thesis focused on the case of two 


observation sequences. 


The multirate optimal filter is an extension of the Wiener—Hopf optimal filter for 
a single input sequence. The use of multiple observation sequences resulted in a lower 
MSE than the basic Wiener—Hopf filter. For this non—adaptive case, the advantage of 
using both data sequences over one alone, the amount of training data needed, and 


observations on optimizing the filter coefficients was addressed. 


The multirate LMS filter is an adaptive filter based on the LMS algorithm of 
Widrow and Hoff. For this adaptive case, results for broadband and narrowband 
interference in an adaptive noise cancellation scenario were given. Results show that 
both narrowband and broadband interference may be removed using the low-rate 


observations alone. 


1. Optimal Filtering 


The performance of the multirate Wiener filter was compared to that of the 
single-rate Wiener filter. Since the MSE while using the multirate Wiener filter with 
both observation sequences is consistently lower than that when using either data set 
alone, an advantage that can be gained when including more than a single observation 


signal. 


The length of training data needed to solve for the filter parameters was 
investigated. The required length of training data was found to be comparable to that 


when using either data sequence alone. Specifically, for the multirate case, solving for K 
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times as many coefficients required a length of training data comparable to that when 
using either the high or low-rate sequence alone. In other words, the increase in number 


of parameters does not translate into an extremely large amount of training data. 


A parametric repesentation of the filter impulse response was investigated and 


seems possible for high orders of the filter, but this was not further pursued. 


2. Adaptive Filtering 


The multirate LMS adaptive filter was shown to perform well for adaptive 
filtering in the context of adaptive noise cancellation. The advantage gained from using 
the filter is the ability to estimate a signal at the full rate using observations taken at a 
lower sampling rate. In an adaptive noise cancellation scenario, the multirate LMS filter 
adequately removed (narrowband) sinusoidal and (broadband) voice interference from a 


recorded speech signal. 


B. RECOMMENDATIONS FOR FUTURE WORK 


The use of multirate data and multirate systems is an important part of modern 
digital signal processing. This thesis has investigated one aspect of the problem, namely 
filtering and estimation, using a statistical approach. A number of specific extensions of 


this work are possible and would be beneficial. 


Finding a parametric representation of the multirate filter impulse response may 
be possible; however, this would be a final research step prior to implementation. If the 
signals could be processed to conform to a known model, then an optimization scheme 
based on the model might be used, as long as it does not add significantly to the 


computational cost. 


Other forms of the optimal filter could be investigated. A recursive least squares 
(RLS) form would be a good next step since it is an adaptive method based on least 
squares [6]. Also, investigation of lattice forms for these filters may give valuable insight 


into the filtering process. 
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In addition to these specific topics, other work is currently ongoing in Professor 
Therrien’s research groups on other aspects of statistic multirate problems. The future 
work will include applications to detection and classification as well as two—dimensional 


(image) signal processing. 
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APPENDIX 


This appendix discusses how the simulation data for this thesis was generated. All 
simulations were performed in MATLAB. Details are given for the methods used to 


generate both the observed and desired signals. 


A. GENERATION OF OBSERVED SIGNALS 


Figure 16 illustrates how data was generated for all simulations of multirate 


Wiener—Hopf filtering. 


Desired signal 
> din 


Observed high-rate signal 


se > x[n] 


PN SEQ 1 
uN Observed low-rate signal 
ace, | | UK VK > y[m] 
PN SE Q 2 Low-pass Decimate 


Figure 16. Diagram of data generation process for simulation of multirate Wiener—Hopf 
filtering. 


A desired signal sequence d(n) of length LZ, was generated (see Section B 
below). The power of the desired signal P, was calculated either from the data or from 
the theoretical formula (see Section B below). The noisy observations sequences were 
created as follows: the high—rate observation sequence x(7) was created by adding a 
pseudorandom Gaussian noise sequence to d(n). This noise sequence is labeled ‘PN 


SEQ 1’ in Figure 16. The power of the noise sequence was calculated to ensure that x(7) 
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had the desired signal-to-noise ratio of SNR,,,,, (see Section C below). The low-rate 
sequence was generated by adding a second (unique) pseudorandom noise sequence, to 
the desired signal d(n) (labeled ‘PN SEQ 2’ in Figure 16). The power of the noise 
sequence was calculated to ensure that the sequence had the desired signal-to-noise ratio 


of SNR,,,,. The sequence was then sent through a low-pass, anti-aliasing filter with a 


low * 
(normalized) cut—off frequency of 1/K (see Section D below). The sequence was then 
decimated by extracting every K“ point to form the low-rate observation sequence 
y(n). 

The reference signals for ANC in the multirate LMS experiments were generated 
by running the interference through third order FIR filters with randomly selected 
coefficients. Two separate FIR filters were created, whose coefficients were random 
numbers uniformly distributed between 1 and 10. Thus, the coefficients were different 
for each trial of the multirate LMS experiment. The high-rate interference was run 
through one filter to produce the high-rate reference signal. The same interference was 


run through the other filter and decimated to produce the low-rate observations. 


B. GENERATION OF THE DESIRED SIGNALS 


The desired signals for this thesis were the AR, FM, and SINE signals, and a 
recorded speech signal. The AR signal was obtained by sending a pseudorandom noise 


sequence of variance o,, = 0.2 through a single—pole IIR filter with a denominator 


polynomial of 1— pz ' where p= 0.95 with a sampling rate of 1 Hz. The theoretical 
power of this signal is P,, = im jo: / (1 —| pl )| [6]. The FM signal was a generated from 
the equation A cos(2 770.041 + cos(0.3¢)) where A =10 with a sampling rate of | Hz. The 
theoretical power of this signal is P,,, = A’/2. The SINE signal was generated from the 
equation Bcos(270.1t) with B= /100 with a sampling rate of 3.33 Hz. The theoretical 


power of this signal is P,,,, = B° / 2. The speech signal was sampled at 44.1 kHz and 


imported into MATLAB in .WAV format using the wavread function. After 
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processing, all relevant signals were saved in .WAV format using the wavwrite 


function. 


C. CALCULATION OF THE SIGNAL-TO-NOISE RATIO 


The following logic and procedure was used to produce signals at the desired 


SNR for these experiments. The SNR in dB for any signal is 


P, 
SNR, =10log,, Gal (30) 


noise 


where the power of any signal g is given by 


P= lola. G1) 


n=0 


For a Gaussian—distributed (normal) noise sequence, the power of the noise sequence is 


its variance. 


A pseudorandom noise sequence was generated using the randn function in 
MATLAB. The function produces a sequence of numbers with a distribution that is 


approximately distributed as N(0,1), i.e., normally distributed with mean zero and 


variance one. If the desired SNR is SNR, and the signal power is P,,,,, then the desired 


signal 


variance of the noise is o~,.. = Fae / 10°") The noise sequence N[n] from the 


randn function was multiplied by the desired standard deviation to produce a noise 


sequence with the correct power 


nn) = (fore) MUI. (32) 


In order to reproduce results, the seed of the normal random number generator was 
specified. The two seeds used in these experiments were [362436069; 521288629] 
for the high-rate data, and [2683551084; 3690929594] for the low-rate data. 
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D. CALCULATION OF ANTI-ALIASING FILTER COEFFICIENTS 


For large sampling rate ratios, the antialiasing filter must have a sharp cutoff in 
order to prevent attenuation of important signal information. With this motivation, the 


anti-aliasing filter was designed so that the transition bandwidth Af is always 10 percent 


of the passband width f 


pass * 


The filter was designed for a positive frequency spectrum 


from 0 to 1 (normalized). Using these constraints, an FIR filter was designed using the 
Remez exchange algorithm in MATLAB. The parameters were: the (normalized) 


top Was 1/ K,, the (normalized) passband frequency /,,,,, Was 


stopband frequency ff, 
(0.9) f,,.»» the passband ripple was 1 dB, and the stopband attenuation was 100 dB. 


Figures 17 and 18 list the coefficients of the FIR antialiasing filters F[m] used in this 


thesis (for K =3 and K =10), for those who wish to reproduce the results herein. The 
filter output is given by 


yn] =>) Flin —i]. (33) 
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Figure 17. 
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Figure 18. (continued). 
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